A topological approach to neural complexity 
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Considerable efforts in modern statistical physics is devoted to the study of networked systems. 
One of the most important example of them is the brain, which creates and continuously develops 
complex networks of correlated dynamics. An important quantity which captures fundamental 
aspects of brain network organization is the neural complexity C(X) introduced by Tononi et at. 
This work addresses the dependence of this measure on the topological features of a network in the 
case of gaussian stationary process. Both analytical and numerical results show that the degree of 
complexity has a clear and simple meaning from a topological point of view. Moreover the analytical 
result offers a straightforward and faster algorithm to compute the complexity of a graph than the 
standard one. 

PACS numbers: : 89.75. He, 87.80.Tq, 89.75.Fb 



I. INTRODUCTION 

The study of networked systems such as the Internet, 
social networks and biological networks has recently at- 
tracted great interest within the statistical physics com- 
munity. A large variety of techniques and models have 
been developed in order to understand or predict the be- 
havior of these systems. Great efforts have been applied 
in discovering their topological features P, 0, 0j S and 
how these properties influence the behaviour of dynam- 
ical processes taking place on them. For example, we 
would like to know how the topology of social networks 
influences the spread of information how the search 

engines are affected by World Wide Web structure 0, |1| ■ 
In this paper we focus on a first basic approach for 
studying the interplay between dynamics and topology 
of brain networks. This study has great interest from sev- 
eral points of view: the brain and its structural features 
can be seen as a prototype of a physical system capable 
of highly complex and adaptable patterns in connectiv- 
ity, selectively improved through evolution; architectural 
organization of brain cortex is one of the key features of 
how brain system evolves, adapts itself to the experience, 
and to possible injuries. 

Brain activity can indeed be modelled as a dynamical 
process acting on a network; each vertex of the structure 
represents an elementary component, such as brain areas, 
groups of neurons or individual cells. A measure, called 
complexity, has been introduced Q with the purpose to 
get a sensible measure of two important features of the 
brain activity: segregation and integration. The former 
is a measure of the relative statistical independence of 
small subsets; the latter is the measure of statistical de- 
viation from independence of large subsets. 
Complexity is based on the values of the Shannon en- 
tropy calculated over the dynamics of the different sized 
subgraphs of the whole network. It is sensitive both to 
the statistical properties of the dynamics and to the con- 
nectivity. 

It has been shown jlo| . by means of genetic algorithms. 



that the graphs showing high values of complexity are 
characterized by being both segregated and integrated; 
the complexity is low when the system is either com- 
pletely independent (segregated), or completely depen- 
dent (integrated). This general behaviour is valid over a 
wide range of dynamical processes 0, 0, 0, 0| . 

Despite this evidence, analytical results about the 
dependence of complexity on the topology and the 
dynamics is still lacking. 

In the following will be proposed a first approach to this 
problem when the dynamics is gaussian. The use of the 
gaussian dynamics get the statistical measure of com- 
plexity independent from the dynamics itself. It doesn't 
pretend to represent any realistic brain structure or 
activity, but to offer a first basic step for understanding 
the relation existing between values of complexity and 
topological properties of brain structure. For this reason 
we used a simplified version of the model introduced 
by E3- This is a first step which could be furtherly 
developed for example for directed and weighted graphs. 

II.DYNAMICS, ENTROPY AND COMPLEXITY 

We consider a graph composed by n vertices and m 
links. It can be represented by its adjacency matrix A, 
whose elements we set to 1 if there is a link between 
the vertices i and j, and otherwise. Only non self con- 
nections are considered and = aji. 
On the graph we model the activity as a stochastic pro- 
cess in the following way: each node i at time t can be in 
a particular state defined by the quantity Xi{i). Given 
our graph with n nodes, the states of the whole graph at 
time t is given by the n — dimensional vector X(t). 

The evolution of X(t) is given by the following dynam- 
ics: 
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where C = A/n and R(i) is an n — dimensional vector 
whose components Ri(t) are random values. Ri{t) is cho- 
sen to be a white gaussian noise, i.e. with the following 
properties : 



= 



R,{t)Rj{t') = a6,,6{t~t') 



(2) 



Here the bar represents the average over the ensemble. 
The normalization n of the adjacency matrix assures that 
the process will reach a stationary state. The dynamics, 
described by eq.l^, is indeed a random walk which is 
damped if the matrix has eingeinvalues |A| < 1. In such 
a way the dynamics reaches a stationary state with a car- 
acteristic time t sa 1/A„ii„, where Xmin is the smallest 
eingenvalues of the matrix C — I. 

It is worth noting that the equation represents a sim- 
plified version of the dynamics introduced in [Tl] | . In that 
case it was considered a gaussian dynamics on directed 
and weighted graphs and with some limitations on the 
values of the variances of each unit (node). 

Since X(f) is a multidimensional gaussian process, its 
statistics is completely described through its second order 
moment: 



X(< + l)X*(t + 1) = CX(t)X*(t)C* + I 



(3) 



X(t -I- l)X*(t + 1) is the n X n covariance matrix whose 
determinant will be referred in the following as |co?;(X)|. 
The average value of X(t) is always zero being a sum of 
zero mean values at each time step. 
Since the process X(t) is gaussian, it is possible to 
show that the Shannon entropy H(X.) depends only on 
\cov{X)\ 



i/(X) = 0.5-ln[(27re)"|coz;(X) 



(4) 



variance of R(t), i.e. |COV(Xi)| = cr"; the term for 
A: = n is instead always null. In the following, we will set 
for the sake of simplicity a = 1. 

In what follows wc will try to find a relation between 
the topology of the graph and its values of Entropy 
i?(X) and Complexity C(X), having defined on it the 
multidimensional gaussian process 

Under stationary conditions, the generic element of 
coi;(X) in the eigenvectors base is: 



A. Ai 



(6) 



The set of value Ai represents the eigenvalue spectrum 
of the adjacency matrix A: 



(7) 



Following (jSJ the determinant of the covariance matrix 



>{X)\=\{ 



1 - 



(8) 



This expression shows that the dynamics depends only 
on the properties of the adjacency matrix A through its 
eigenvalue spectrum. As a consequence the statistical 
properties of the stationary states can be analized with- 
out studying their time evolution but by looking at their 
eigenvalue spectrum. On the other hand the richness in 
information embedded in the eigenvalue spectrum makes 
the analysis not trivial at all The aim of the next 

paragraph is to show which topological properties cm- 
bedded into the spectrum dominate the behaviour of the 
dynamical process. 



Let us consider all the possible subgraphs of rank k 
(number of nodes) of the whole graph. Each of these 
subgraphs are indicated as X^. 

The complexity has been defined as: 



III. CONNECTION WITH THE TOPOLOGY 

Using the equation ©, H(X.) becomes: 



(H(Xfc)) - -H{X) 
n 



(5) 



where the average (...) is taken over all the subgraphs 
of rank k. The sum ranges from the minimum possible 
rank of a subgraph, i.e. 2 to n — 1. The term in (jSJ 
for fc = 1 would be trivial since the covariance matrix 
of disconnected vertices is simply dependent only on the 



H(X.) = 0.5-ln[(27re)"|cow(X)| 

1 



= 0.5-^ In 



(27re)- 



1 



(9) 



If A^Q^/r?^ << 1, we can consider the following series 
expansion: 
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and by substitution in we get: 



i/(X) ln(27re)t +0.5(^1+^^ 



(10) 



C(X) 
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since: 



Ea. Hll|l allows us to relate i?(X) to the number Dk; 
this is the number of fc — step directed paths of the under- 
lying -undirected- graph, which return to their starting 
node after k steps: 



(12) 



41, 42, 



where ai^^i^^-^ is the generic non zero element of the 
adiancency matrix A. 



Using this result H{X.) becomes: 
H{X.) ~ ln(27re; 



0.5 



D2 

n 2 



2n4 



of A 



(13) 



D2 is then the number of paths which starting from 
any node i go to any other one j and then come back to 
i. Remembering that an unconnected pair of nodes has 
aij = 0, D2 is obviously twice the number of links of the 
whole graph. 

Thus the first two terms of -ff (X) expansion depend 
only on the number of nodes and links, and not on the 
graph topology. 



{m(k)) = m 



k{k - 1) 
n{n — 1) 



So far, the value of C(X) is not defined by the topology. 
In order to reveal something related to a particular link's 
arrangement, we need to consider the further terms in the 
expansion. We can rewrite the complexity C(X) in the 
following way to put in evidence the part dependent only 
on the number of links m and nodes n of the whole graph 
(and so independent from the topology), C°'^'^'^{n,m): 



C(X) = C°''''^(n,m) + 



4- 



k=2 ^ 



RiX) = O 
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where we have explicitly written the term in 



In order to express the topological information con- 
tained in the eo. lfT^ . let us consider that 



{D,{k)) = (^A,(fc)4) 



{4m{kf)-J2{MkfHkf) (17) 



and 



Consider now the value of complexity C(X) up to the 
D2 term in the entropy. We get 



^ A2(fc)A2(fc) = ^ X! auauajgagj - 2D2{k) (18) 
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From ea. H17() we see that the C(X), at this order of 
approximation, depends on the second order moment of 
the number of hnks (m(/c)^), calculated over all the sub- 
graphs of rank k (we remember that a subgraph of rank 
fc is a particular choice of k nodes in the whole graph 
and fc G [2,71 — 1]). This is the first quantity dependent 
on the topology that we can easily evaluate on the 
graph in place of the original expression of complexity. 
In the next step we will show that the fourth order 
approximated value of complexity can be expressed 
through (m(fc)^) calculated over all the subgraphs of 
rank k, (i.e. fc £ [2,n — 1]) and the second order moment 
of the degree distribution {{q^)) of the whole graph. This 
will allow to distinguish the complexity of graphs with 
the same total number of nodes n and total number of 
links m, through the evaluation of less time consuming 
measures than the complexity. Moreover this result will 
offer a deeper understanding of what the complexity 
measure means from the topological point of view. 

The terms in the sum of ea. (ll8ll can be explicitly ex- 
pressed as: 



J2i^j;i=j,q=ii'^ij'^ji)'^ + J2i^j^i^q auauajqaqj-] 
3 auaiiUjiaij + 2 aiiauajjajj + 



+ '}li=jtj;l=i,q=j O'li'^ii'^jjO-jj + 2 X] 



(19) 

In the expression (|19() . only the first three terms are 
non zero, while the others contain at least a diagonal 
element an = 0. Moreover the first term is just D2. 

It is easy to show that the second and the third terms 
in the sum correspond to the number of paths ("loops") 
of the type shown in Fig. ^ 




generic node i, i.e. the node i has q links. 

Consider now a particular subgraph of rank k. It has k 
nodes, each of them having a certain number of links or 
none. Consider then all the nodes having the same degree 
g in a generic subgraph of rank k; then the number of 
paths (loops) of the first type in Fig^ involving this kind 
of nodes are: 



n — 3 
fc - 3 



where 



is the number of ways to choose k nodes 



over a total of n nodes, leaving aside the 3 nodes which 
belong to the pair considered. This is the number of 
subgraph of rank k, in the whole graph, which contain a 
particular choice of 3 nodes. Moreover, since the generic 
node i has q links, we can count (2) different pairs of 
links sharing the same node i. 

If we denote with P{q) the degree distribution in the 
whole graph, then we can write the third term in cq. I|19|) 
as : 



E 



diiauajiaij 



{fc} iv^J^i 



n — 3 
fc-3 



P{q)dq 



n — 3 
fc-3 



im)-{m 



where we explicitely wrote the X]{fe} 



^ ^ which we will use 
later. X]{fc} means the sum over all the subgraphs with 
k nodes or of rank fc. 

Consider now the second term in ea. (|19|l . i.e. 
J^i^j^k^q'^ikO'kiO-jqaqj- This is the number of disjoint 
pairs of links in a generic subgraph of rank fc. 
To compute such number, we first count the total number 
of pairs in the whole graph, i.e. (™) ; then we subtract 
the number of pairs of links sharing a node in the whole 
graph (from the previous computation) . Finally we have 
to consider the multiplicity (^^4) 1 i-C- the number of sub- 
graphs of rank fc containing each pair of disjoint links. 
Then the second term in eq. (|19|l is (again considering also 
the sum: Y^^^^) : 





FIG. 1: top: paths described by the third term in eg. 1191 : 
bottom: paths described by the second term in eg. 1191 

We will show that the number of these paths is related 
to (((?^)). In what follows q represents the degree of the 



E E auauajqaqj 

{k} t^J^l^q 



m 
2 

n - 4 
fc-4 



Remembering that in ea. (|15|l wc have to compute 
the quantity X]a:=2 (^4^^))' then we have to evaluate 
J2kZlJ2i^jJ2iq{auaHa.jqaqj}. The average (...) is per- 
formed for a particular value of fc over all the subgraph of 
rank fc. For this reason in previous expressions we have 
explicitely written the sum over all the subgraphs of 
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rank fc, i.e. X]{fe}- To perform the average (...) we have 
then simply to divide such expressions for the number of 
subgraph of rank k contained in the whole graph, i.e.(^). 
Eventually wc have to sum over all the values of k, Y^f.- 

Then the final expression for X]a;(^4(^)) becomes: 






Regular 



Small World 



Random 



(20) 



where the second and third term of ea. (|19|l have been 
substituted with their explicit computations. 
It is worth to note that Y^^=2^{l)) only a function of 
n and m, being {{q)) = ^. 

In the following is the whole expression of C{X): 



k=2 ' 



+C^-''\m,n,{{q^)))+C^-^\m,n,{{q))) + 

~iord4:f 



i{kf))~ 



+CZ'^'^{m,n) + R{X) 



(21) 



where C°^'^^{n,m) and i?(A) are respectively the 
ea. ()14|l and ca. H16(l . The other terms are: 



Cr\m,n,Y.lZl{m{k?)) = Till ^ 



2\\\ _ {{q )) 



Cr''^(m,n,((g2))) 



{{q^)) ( 1 (rt-4)!fc! _ ^^n-l 3 (n-3)!fc! 
8 \2^k=4: fe4 (fc_4)!„! Z^fc=3 fci (fc-3)!Ti! 
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FIG. 2: 3 difFerent network models: a regular network, the 
small word model and a random network. It is possible to 
go from one model to the other varying the probability p of 
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FIG. 3: Behaviour of complexity measure and its approxi- 
mation (up to the fourth order) in a small world graph with 
n=10,15,20 nodes and p=0.1, against J, i.e. number of first 
neighbors. 



IV. NUMERICAL RESULTS 

We perform calculation of entropy and complexity of 
the dynamics ^ over a small world graph with n = 
10, 15, 20 nodes. We estimate both the exact and the ap- 
proximate values, for checking the accuracy of the ap- 
proximation, and their dependence on the topological 
properties of the graphs. 

The algorithm behind the model can be summarized 
in two steps 0|: 

(1) Start with a ring lattice with n nodes in which every 
node is connected to its first J neighbors {J / 2 on either 
side). In order to have a sparse but connected network 
at all times, consider n » J » ln{n) >> 1. 

(2) Randomly rewire each edge of the lattice with proba- 
bility p such that self-connections and duplicate edges are 
excluded. Varying p the transiton between order (p = 0) 
and randomness {p = 1) can be closely monitored (FiglSJ. 

The numerical evaluation for the exact and the ap- 
proximated values of C(X) can be easily achieved in a 
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FIG. 4: Behaviour of complexity measure and its approxi- 
mation (up to the fourth order) in a small world graph with 
n=20 nodes and 3=4, against p, probability of rewiring. 



small world graph: in this case we can investigate dilfer- 
ent arrangements of links keeping the number of nodes 
and links fixed. The variation of complexity is affected 
both by {m{k)^) over all the scales fc, and ((g^))- 

Since the expansion is allowed when the average node 
degree is much less than one (A^^^/n^ << 1), we expect 
a higher accuracy when the average connectivity is low 
(n » J), and a worse approximation when J increases. 
The simulations confirm this trend for increasing values 
of J, and J <n (Fig. OJ. 

In Fig^lwe show the exact and approximated behaviour 



of C(X) versus the probability of rewiring. Their relative 
difference is much less than one and they show very 
similar behaviour. 

Analogous results have been found for the other values 
of n. 

V. DISCUSSION 

We attempted to extract the topological meaning of 
the complexity measure in the case of gaussian dynamic. 
This aim has been achieved both from analytical and 
numerical points of view, showing that very good 
approximation of complexity can be obtained through 
two simple direct topological measures on the graph, 
namely, the second order moment of the number of 
links {rn^ik)) over all the scales k, and the second 
order moment of the node degree ((9^)) on the whole 
graph. The analytical expression is obtained through 
an expansion for Xmax/n << 1; however the numerical 
results show the expression (|21|) for the complexity is a 
reasonable approximation even for Xmax/n ~ 1. 
The relevance of the obtained results relies on two main 
aspects: the measure has a clear topological meaning 
which help to understand in a more intutive way the 
degree of complexity of a graph; it can be evaluated 
through two less time-consuming, and considerably 
easier to compute topological measures. The saving of 
computation time is of order n since evaluating the two 
mesures mentioned above require steps instead of the 
steps of the diagonalizing algorithms for symmetric 
matrices. 

We enjoyed useful discussions and suggestions by 
Dr.V.Scrvedio and Dr. A. Capocci. We would like to 
thank Dr. F.Colaiori for her help in drawing figures. 
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